120 research outputs found
A survey on mouth modeling and analysis for Sign Language recognition
© 2015 IEEE.Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR
The Conditional Lucas & Kanade Algorithm
The Lucas & Kanade (LK) algorithm is the method of choice for efficient dense
image and object alignment. The approach is efficient as it attempts to model
the connection between appearance and geometric displacement through a linear
relationship that assumes independence across pixel coordinates. A drawback of
the approach, however, is its generative nature. Specifically, its performance
is tightly coupled with how well the linear model can synthesize appearance
from geometric displacement, even though the alignment task itself is
associated with the inverse problem. In this paper, we present a new approach,
referred to as the Conditional LK algorithm, which: (i) directly learns linear
models that predict geometric displacement as a function of appearance, and
(ii) employs a novel strategy for ensuring that the generative pixel
independence assumption can still be taken advantage of. We demonstrate that
our approach exhibits superior performance to classical generative forms of the
LK algorithm. Furthermore, we demonstrate its comparable performance to
state-of-the-art methods such as the Supervised Descent Method with
substantially less training examples, as well as the unique ability to "swap"
geometric warp functions without having to retrain from scratch. Finally, from
a theoretical perspective, our approach hints at possible redundancies that
exist in current state-of-the-art methods for alignment that could be leveraged
in vision systems of the future.Comment: 17 pages, 11 figure
3D face morphable models "In-The-Wild"
3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions (in-the-wild). In this paper, we propose the first, to the best of our knowledge, in-the-wild 3DMM by combining a powerful statistical model of facial shape, which describes both identity and expression, with an in-the-wild texture model. We show that the employment of such an in-the-wild texture model greatly simplifies the fitting procedure, because there is no need to optimise with regards to the illumination parameters. Furthermore, we propose a new fast algorithm for fitting the 3DMM in arbitrary images. Finally, we have captured the first 3D facial database with relatively unconstrained conditions and report quantitative evaluations with state-of-the-art performance. Complementary qualitative reconstruction results are demonstrated on standard in-the-wild facial databases
DenseReg: fully convolutional dense shape regression in-the-wild
In this paper we propose to learn a mapping from image pixels into a dense template grid through a fully convolutional network. We formulate this task as a regression problem and train our network by leveraging upon manually annotated facial landmarks “in-the-wild”. We use such landmarks to establish a dense correspondence field between a three-dimensional object template and the input image, which then serves as the ground-truth for training our regression system. We show that we can combine ideas from semantic segmentation with regression networks, yielding a highly-accurate ‘quantized regression’ architecture. Our system, called DenseReg, allows us to estimate dense image-to-template correspondences in a fully convolutional manner. As such our network can provide useful correspondence information as a stand-alone system, while when used as an initialization for Statistical Deformable Models we obtain landmark localization results that largely outperform the current state-of-the-art on the challenging 300W benchmark. We thoroughly evaluate our method on a host of facial analysis tasks, and demonstrate its use for other correspondence estimation tasks, such as the human body and the human ear. DenseReg code is made available at http://alpguler.com/DenseReg.html along with supplementary materials
3D Reconstruction of 'In-the-Wild' Faces in Images and Videos
This is the author accepted manuscript. The final version is available from IEEE via the DOI in this record 3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and are among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ('in-the-wild'). In this paper, we propose the first 'in-the-wild' 3DMM by combining a statistical model of facial identity and expression shape with an 'in-the-wild' texture model. We show that such an approach allows for the development of a greatly simplified fitting procedure for images and videos, as there is no need to optimise with regards to the illumination parameters. We have collected three new benchmarks that combine 'in-the-wild' images and video with ground truth 3D facial geometry, the first of their kind, and report extensive quantitative evaluations using them that demonstrate our method is state-of-the-art.Engineering and Physical Sciences Research Council (EPSRC
Estimating correspondences of deformable objects "in-the-wild"
This is the author accepted manuscript. The final version is available from IEEE via the DOI in this recordDuring the past few years we have witnessed the development of many methodologies for building and fitting Statistical Deformable Models (SDMs). The construction of accurate SDMs requires careful annotation of images with regards to a consistent set of landmarks. However, the manual annotation of a large amount of images is a tedious, laborious and expensive procedure. Furthermore, for several deformable objects, e.g. human body, it is difficult to define a consistent set of landmarks, and, thus, it becomes impossible to train humans in order to accurately annotate a collection of images. Nevertheless, for the majority of objects, it is possible to extract the shape by object segmentation or even by shape drawing. In this paper, we show for the first time, to the best of our knowledge, that it is possible to construct SDMs by putting object shapes in dense correspondence. Such SDMs can be built with much less effort for a large battery of objects. Additionally, we show that, by sampling the dense model, a part-based SDM can be learned with its parts being in correspondence. We employ our framework to develop SDMs of human arms and legs, which can be used for the segmentation of the outline of the human body, as well as to provide better and more consistent annotations for body joints.Engineering and Physical Sciences Research Council (EPSRC)TekesEuropean Community Horizon 202
Crystal Structure Studies of Human Dental Apatite as a Function of Age
Studies of the average crystal structure properties of human dental apatite
as a function of the tooth-age in the range of 5-87 years are reported. The
crystallinity of the dental hydroxyapatite decreases with the tooth-age. The
a-lattice constant that is associated with the carbonate content in carbonate
apatite decreases with the tooth-age in a systematic way, whereas the c-lattice
constant does not change significantly. Thermogravimetric measurements
demonstrate an increase of the carbonate content with the tooth-age. FTIR
spectroscopy reveals both, B and A-type carbonate substitutions with the B-type
greater than the A-type substitution by a factor up to ~5. An increase of the
carbonate content as a function of the tooth-age can be deduced from the ratio
of the v2 CO3 to the v1 PO4 IR modes.Comment: 17 page
Trained Immunity Confers Prolonged Protection From Listeriosis.
Trained immunity refers to the ability of the innate immune system exposed to a first challenge to provide an enhanced response to a secondary homologous or heterologous challenge. We reported that training induced with β-glucan one week before infection confers protection against a broad-spectrum of lethal bacterial infections. Whether this protection persists over time is unknown. To tackle this question, we analyzed the immune status and the response to Listeria monocytogenes (L. monocytogenes) of mice trained 9 weeks before analysis. The induction of trained immunity increased bone marrow myelopoiesis and blood counts of Ly6C <sup>high</sup> inflammatory monocytes and polymorphonuclear neutrophils (PMNs). Ex vivo, whole blood, PMNs and monocytes from trained mice produced increased levels of cytokines in response to microbial products and limited the growth of L. monocytogenes. In vivo, following challenge with L. monocytogenes, peripheral blood leukocytes were massively depleted in control mice but largely preserved in trained mice. PMNs were reduced also in the spleen from control mice, and increased in the spleen of trained mice. In transwell experiments, PMNs from trained mice showed increased spontaneous migration and CXCL2/MIP2α-induced chemotaxis, suggesting that training promotes the migration of PMNs in peripheral organs targeted by L. monocytogenes. Trained PMNs and monocytes had higher glycolytic activity and mitochondrial respiration than control cells when exposed to L. monocytogenes. Bacterial burden and dissemination in blood, spleen and liver as well as systemic cytokines and inflammation (multiplex bead assay and bioluminescence imaging) were reduced in trained mice. In full agreement with these results, mice trained 9 weeks before infection were powerfully protected from lethal listeriosis. Altogether, these data suggest that training increases the generation and the antimicrobial activity of PMNs and monocytes, which may confer prolonged protection from lethal bacterial infection
Explicit Evidence Systems with Common Knowledge
Justification logics are epistemic logics that explicitly include
justifications for the agents' knowledge. We develop a multi-agent
justification logic with evidence terms for individual agents as well as for
common knowledge. We define a Kripke-style semantics that is similar to
Fitting's semantics for the Logic of Proofs LP. We show the soundness,
completeness, and finite model property of our multi-agent justification logic
with respect to this Kripke-style semantics. We demonstrate that our logic is a
conservative extension of Yavorskaya's minimal bimodal explicit evidence logic,
which is a two-agent version of LP. We discuss the relationship of our logic to
the multi-agent modal logic S4 with common knowledge. Finally, we give a brief
analysis of the coordinated attack problem in the newly developed language of
our logic
Long-range transfer of electron-phonon coupling in oxide superlattices
The electron-phonon interaction is of central importance for the electrical
and thermal properties of solids, and its influence on superconductivity,
colossal magnetoresistance, and other many-body phenomena in
correlated-electron materials is currently the subject of intense research.
However, the non-local nature of the interactions between valence electrons and
lattice ions, often compounded by a plethora of vibrational modes, present
formidable challenges for attempts to experimentally control and theoretically
describe the physical properties of complex materials. Here we report a Raman
scattering study of the lattice dynamics in superlattices of the
high-temperature superconductor and the
colossal-magnetoresistance compound that suggests
a new approach to this problem. We find that a rotational mode of the MnO
octahedra in experiences pronounced
superconductivity-induced lineshape anomalies, which scale linearly with the
thickness of the layers over a remarkably long range of
several tens of nanometers. The transfer of the electron-phonon coupling
between superlattice layers can be understood as a consequence of long-range
Coulomb forces in conjunction with an orbital reconstruction at the interface.
The superlattice geometry thus provides new opportunities for controlled
modification of the electron-phonon interaction in complex materials.Comment: 13 pages, 4 figures. Revised version to be published in Nature
Material
- …